The present invention relates to a robot which is used in house and has a function to store an image(s) of a user""s face and identify a person by using image information, an identifying environment determining method, and a program thereof.
In recent years, a robot typified by a pet-type robot has been developed for entertainment, which is configured so as to pick tip and register a face image of its owner, identify a person by using the picked up image, and call his/her name, and so forth.
FIG. 1 is a diagram showing an example of a configuration of a conventional robot having a face identifying function. The robot comprises an imaging means 1, a face detecting and identifying means 2, an information informing means 3, an information inputting means 4, a controlling means 7, a moving means 8 or the like. The imaging means 1 picks up video. The face detecting and identifying means 2 detects, from an image obtained at the imaging means 1, a face area of a person and stores it, and identifies a person with a newly obtained image using stored face image information. The information informing means 3 informs a user about result of identification obtained at the face detecting and identifying means 2. The information inputting means 4 is disposed so that a user can input whether result of identification informed to the user by the information informing means 3 is correct or incorrect. The controlling means 7 is disposed for controlling the whole robot. The moving means 8 is disposed so that the robot itself can move. The robot communicates with a user, on the basis of commands from the controlling means 7, through the information inputting means 4 and the information informing means 3. Further, the robot moves in a room using the moving means 8. Subsequently, the robot recognizes a user""s face using the imaging means 1 and the face detecting and identifying means 2, and determines who is the user dialoguing with the robot face-to-face.
In the case of face identification using a robot for domestic use, a lighting environment in a house is of a problem. In a house, it is often the case that a room is equipped with one light and that a house is designed so as to let in much sunshine. Therefore, the amount of lighting changes drastically according to places and time zones. Moreover, compared to offices where lighting is designed for fine work, lighting in a house is relatively dark. Therefore, there may exist an environment not suitable for identification significantly. In a conventional identifying system, it is difficult to absorb variations of amounts of characteristics caused by the variations of lighting under such conditions. This leads to deterioration of accuracy of identifying performance. Furthermore, when a face area is not illuminated sufficiently, it is difficult to extract characteristics required for identification, which leads to failuer of identification.
To execute face identification under such conditions, there is disclosed a technique in Japanese Patent Application Laid-Open No. 2000-222576. The technique is intended to absorb the variations of lighting by storing, as dictionary data, an amount of characteristics extracted from a plurality of images, per person, each in which the way to illuminate the person is different, and executing linear discrimination using the data.
However, even by this method, it is difficult to absorb the variations between a face image obtained at registration and that obtained under such condition that the lighting state is remarkably different from that at the registration, which leads to incorrect identification.
It is therefore an object of the present invention to provide a robot, an identifying environment determining method, and a program thereof, wherein it is possible to reduce incorrect identification (matching) in the case of executing face identification in a place where variations of lighting are large such as in a house and in a place where a lighting environment is significantly bad for identification.
According to a first aspect of the present invention, for achieving the objects mentioned above, there is provided a robot comprising:
an imaging means for picking up an image and digitizing the image;
a face detecting and identifying means for detecting, from an image acquired at the imaging means, a face area of a person and storing the face area, and identifying a person using a stored image and face image information that is referred for identification;
an information informing means for informing a user about result of identification acquired at the face detecting and identifying means;
an information inputting means disposed so that a user can input information whether result of identification informed to the user by the information informing means is correct or incorrect;
an identification result reliability calculating means for determining, using information from the imaging means, whether or not a present lighting state is suitable for identification;
a controlling means for controlling a robot; and
a moving means for moving a robot.
According to a second aspect of the present invention, in the first aspect, the identification result reliability calculating means determines a lighting environment using a variance value of pixel values in a face area detected at the face detecting and identifying means.
According to a third aspect of the present invention, in the first aspect, the identification result reliability calculating means determines a lighting environment using a ratio of an average pixel value in a face area detected at the face detecting and identifying means to that in the vicinity of an outside of the face area.
According to a fourth aspect of the present invention, in the first aspect, the identification result reliability calculating means determines a lighting environment using an average pixel value of whole image.
According to a fifth aspect of the present invention, in the first aspect, the robot further comprises
a robot state observing means observing a lighting state in a room using a lighting device state observing means working with a lighting device in a room to detect whether or not the lighting device is turned on, wherein
the identification result reliability calculating means determines a lighting environment using information whether or not the lighting device is turned on.
According to a sixth aspect of the present invention, in the first aspect, the robot further comprises
a robot state observing means observing a curtain state in a room using a curtain state observing means working with curtains to detect whether or not the curtains are drawn, wherein
the identification result reliability calculating means determines a lighting environment using information whether or not the curtains are drawn.
According to a seventh aspect of the present invention, in the first aspect, the robot further comprises
a robot state observing means retaining information about time, wherein
the identification result reliability calculating means determines a lighting environment using information about time before executing identification.
According to an eighth aspect of the present invention, in the first aspect, the robot further comprises
a robot state observing means retaining positional information acquired by observing a position of a robot in a room, wherein
the identification result reliability calculating means determines a lighting environment using the positional information.
According to a ninth aspect of the present invention, there is provided a robot comprising:
an imaging means for picking up an image and digitizing the image;
a face detecting and identifying means for detecting, from all image acquired at the imaging means, a face area of a person and storing the face area, and identifying a person using a stored image and face image information that is referred for identification;
an information informing means for informing a user about result of identification acquired at the face detecting and identifying means;
an information inputting means disposed so that a user call input information whether result of identification informed to the user by the information informing means is correct or incorrect;
a lighting environment determining means for determining a lighting environment on the basis of image information acquired from the imaging means, and determining which predetermined lighting category corresponds to the lighting environment;
an identifying performance recording means for recording, as identifying performance information, a classification of a lighting category at face identification, the category being determined at the lighting environment determining means, number of times where face identification has succeeded per lighting category, and number of times where face identification has been executed per lighting category;
a lighting environment comparing means for determining, on the basis of the identifying performance information recorded in the identifying performance recording means, whether or not a present lighting environment is suitable for identification;
a controlling means for controlling a robot; and
a moving means for moving a robot.
According to a tenth aspect of the present invention, there is provided a robot comprising:
an imaging means for picking tip an image and digitizing the image;
a face detecting and identifying means for detecting, from an image acquired at the imaging means, a face area of a person and storing the face area, and identifying a person using a stored image and face image information that is referred for identification;
an information informing means for informing a user about result of identification acquired at the face detecting and identifying means;
an information inputting means disposed so that a user call input information whether result of identification informed to the user by the information informing means is correct or incorrect;
a lighting environment determining means for determining a lighting environment on the basis of image information acquired from the imaging means, and determining which predetermined lighting category corresponds to the lighting environment;
an identifying performance recording means for recording, as identifying performance information, a classification of a lighting category at face identification, the category being determined at the lighting environment determining means, number of times where face identification has succeeded per lighting category, and number of times where face identification has been executed per lighting category;
a lighting environment comparing means for determining, on the basis of the identifying performance information recorded in the identifying performance recording means, whether or not a present lighting environment is suitable for identification;
a controlling means for controlling a robot;
a moving means for moving a robot; and
a moving plan making means for making a moving plan, on the basis of information recorded in the identifying performance recording means, to move a robot to a position where identifying performance is higher.
According to an eleventh aspect of the present invention, there is provided a robot comprising:
all imaging means for picking up an image and digitizing the image;
a face detecting and identifying means for detecting, from an image acquired at the imaging means, a face area of a person and storing the face area, and identifying a person using a stored image and face image information that is referred for identification;
an information informing means for informing a user about result of identification acquired at the face detecting and identifying means;
an information inputting means disposed so that a user call input information whether result of identification informed to the user by the information informing means is correct or incorrect;
a lighting environment determining means for determining a present lighting environment on the basis of image information acquired from the imaging means, determining which predetermined lighting category corresponds to the lighting environment, and classifying the lighting environment of a face image stored in the face detecting and identifying means into a lighting category;
a lighting environment at registration recording means for recording, per user, the lighting category of the stored face image;
a lighting environment at registration comparing means for determining, on the basis of information recorded in the lighting environment at registration recording means, whether or not a face image of a user communicating with a robot has been registered in a same lighting environment as a present lighting environment;
a controlling means for controlling a robot; and
a moving means for moving a robot.
According to a twelfth aspect of the present invention, there is provided a robot comprising:
an imaging means for picking up an image and digitizing the image;
a face detecting and identifying means for detecting, from an image required at the imaging means, a face area of a person and storing the face area, and identifying a person using a stored image and face image information that is referred for identification;
an information informing means for informing a user about result of identification acquired at the face detecting and identifying means;
an information inputting means disposed so that a user can input information whether result of identification informed to the user by the information informing means is correct or incorrect;
a lighting environment determining means for determining a lighting environment on the basis of image information acquired from the imaging means, and determining which predetermined lighting category corresponds to the lighting environment;
a face detecting means for detecting, from an image acquired at the imaging means, a face area to extract the face area, and converting the face area into data to execute identification;
a data controlling means for storing a characteristic data group of a face per person at acquisition thereof in each lighting category acquired at the lighting environment determining means;
a dictionary compiling means for compiling a recognition dictionary using characteristic data per lighting category;
a dictionary storing means for storing, per lighting category, a recognition dictionary compiled at the dictionary compiling means;
a dictionary selecting means for selecting, from the dictionary storing means, a recognition dictionary of a same lighting category is that determined at the lighting environment determining means, and supplying the dictionary to a face identifying means;
a face identifying means for executing identification of characteristic data supplied from the face detecting means to determine a user""s name on the basis of a recognition dictionary supplied from the dictionary selecting means;
a controlling means for controlling a robot; and
a moving means for moving a robot.
According to a thirteenth aspect of the present invention, in one of the aspects 9 to 12, the lighting environment determining means categorizes a lighting environment using a variance value of pixel values in a face area detected at the face detecting and identifying means.
According to a fourteenth aspect of the present invention, in one of the aspects 9 to 12, the lighting environment determining means categorizes a lighting environment using a ratio of an average pixel value in a face area detected at the face detecting and identifying means to that in the vicinity of an outside of the face area.
According to a fifteenth aspect of the present invention, in one of the aspects 9 to 12, the lighting environment determining means categorizes a lighting environment using an average pixel value of whole image.
According to a sixteenth aspect of the present invention, in one of the aspects 9 to 12, the robot further comprises
a robot state observing means observing a lighting state in a room using a lighting device state observing means working with a lighting device in a room to detect whether or not the lighting device is turned on, wherein
the lighting environment determining means categorizes a lighting environment using information whether or not the lighting device is turned on.
According to a seventeenth aspect of the present invention, in one of the aspects 9 to 12, the robot further comprises
a robot state observing means observing a curtain state in a room using a curtain state observing means working with curtains to detect whether or not the curtains are drawn, wherein
the lighting environment determining means categorizes a lighting environment, using information whether or not the curtains are drawn.
According to an eighteenth aspect of the present invention, in one of the aspects 9 to 12, the robot further comprises
a robot state observing means retaining information about time, wherein
the lighting environment determining means categorizes a lighting environment using information about time before executing identification.
According to a nineteenth aspect of the present invention, in one of the aspects 9 to 12, the robot further comprises
a robot state observing means retaining positional information acquired by observing a position of a robot in a room, wherein
the lighting environment determining means categorizes a lighting environment using the positional information.
Furthermore, the present invention includes an identifying and determining method for determining a lighting environment and a program for executing the method by using the robot.